In the past, computing applications such as computer games and multimedia applications used controllers, remotes, keyboards, mice, or the like to allow users to manipulate game characters or other aspects of an application. More recently, computer games and multimedia applications have begun employing cameras and software gesture recognition engines to provide a natural user interface (“NUI”). With NUI, a capture device is used to detect user movements, gestures and speech. The associated audio and visual data is forwarded to a processor which interprets the data and uses it to control game characters or other aspects of an application.
Current NUI systems may use a single capture device to register data from a play space or scene. However, it may be desirable to increase the size of the play space or scene. Use of a single capture device in this case may be inadequate for a number of reasons, such as for example inadequate resolution or illumination of the scene, or occlusion of objects within the scene. For such cases, it would be advantageous to use more than one capture device. There is a need for a system to synthesize data from multiple capture devices viewing a common scene to reconcile the data from the multiple sources.